Proxies, Application Interfaces, and Distributed Systems 



Amitabh Dave, Mohlalefi Scfika and Roy IL Campbell 
Department of Computer Science 
University of Illinois at Urban a- Champaign 
Urbana, IL 61820 



Abstract 

Proxy objects are local representatives ofrtmoU ob- 
jects in a distributed system. Wt use proxies to con- 
struct a transparent application programming interface 
(API) for the Choices distributed operating system. Jn 
earlier work, proxies were used in Choices to provide a 
protected, object-oriented interface to system objects. 
The addition of RtmoteProzies allows applications to 
access all resources in a uniform way by simply invok- 
ing methods on objects, irrespective of whether they 
are local, in the kernel, in a different user virtual ad- 
dress space or remote. We also extend proxies as de- 
fined by Shapiro{10] to optimize access to remote and 
protected objects and to provide support for changing 
server interfaces. We describe a new remote procedure 
call (RPC) facility for invoking methods on remote 
objects through the proxy mechanism. The API is 
made dynamically reconfigurable by using table lookup 
to perform all functions normally provided by stubs 
in conventional RPC implementations^!], last, the 
API permits new versions of a service to be introduced 
without requiring recompilation of application client 
code. 



1 Introduction 

An application programming interface (API) pro- 
vides a set of run time facilities and services that 
may be invoked by applications. In a distributed sys- 
tem, these facilities and services are implemented by 
functions and servers in the Verne), in separate vir- 
tual memory spaces, or on remote computers. A dis- 
tributed operating system provides location transpar- 
ent access to remote servers and resources. In this pa- 
per, we argue for a transparent application interface 
that hides whether facilities and set vices ate provided 
as a libiary to the application or provided by the dis- 
tributed system upon which it runs. 

In Choices, we view all entities in a system as ob- 



jects that belong to a class hierarchy. The application 
interface is the mechanism through which application 
objects invoke facilities and services from other server 
and peer objects that may be in the kernel, on the 
local system but in a different virtual memory space, 
or on a remote system. The object's methods may 
be accessed independently of these attributes through 
the transparent API. Untike conventional APIs which 
distinguish between a basic set of services and the ser- . 
vices provided by servers, the Choices API permits the 
set of functions and services that are available to an 
application to be dynamic and uniformly accessible. 

Proxy objects are local representatives of remote 
objects in a distributed system[10). The Choices API 
mechanism is built using proxy objects[2, 9]. In dis- 
tributed Choices, proxy objects represent other ob- 
jects that cannot be accessed directly from an appli- 
cation. Proxy objects provide indirection and late, run 
time binding. A method invocation on a proxy results 
in a corresponding method invocation on the object it 
represents. 

A transparent API for an object-oriented dis- 
tributed operating system has several benefits. Appli- 
cations may be written and compiled to be indepen- 
dent of the location of the servers and other resources 
they access. This simplifies porting and object mi- 
gration. The API separates the provision of resources 
from bow those resources are implemented. The re- 
sources may be implemented in the same user space as 
the application, in a different user space, in the kerne), 
or on a remote machine. The user of an application or 
an operating system load balancing policy may choose 
an appropriate implementation of a service or function 
depending upon load, data set si»e, application host 
hardware, or other considerations. The application 
may be run on a "microkernel" with most resources 
accessed through user level servers or on a more tra- 
ditional kernel with the resources provided as kernel 
services. 

The use of proxy objects provides a transparent ap- 
plication interface and a simple and uniform access 
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model for all objects in the system. In Chc\ct$ i proxy 
objects support the object-oriented notions of class hi- 
erarchy and inheritance. Proxy objects may be used 
to configure dynamically and efficiently the run time 
support for an application. In addition, proxy objects 
allow interfaces to be reused when new versions of the 
classes axe introduced. To support dynamic configu- 
ration and class versions, we have built a new imple- 
mentation of the Choices proxy object that provides 
serveT interfaces with a table driven RPC facility in- 
stead of subroutine stubs. This mechanism and how< 
it allows proxy objects to be used to support an open 
systems object-oriented API are discussed below. 

Existing systems provide transparent application 
interfaces in several ways. For example, in message 
passing systems, ports are location independant com- 
munication entities that represent servers. Restrict- 
ing access to all system resources through these ports 
makes the interface transparent [4], However, applica- 
tions must distinguish between those resources that 
can be reached directly and those that can be ac- 
cessed through the message passing system. Objects 
of the same class that reside on different remote sys- 
tems must be accessed using different ports. Remote 
procedure calls (RPCs) go one step further and hide 
the message passing system. RPCs have been shown 
to work well both for cross-machine calls as well as 
cross-domain calls on the same machine[l]. Current 
implementations of RPCs force applications to link 
with libraries of stub functions which implement the 
actual remote calls. This makes it difficult to change 
server interfaces dynamically. 

The RPC. stubs arc usually built from the defini- 
tions of the procedures. Using RPC support to call 
methods on objects is simple if all the objects of a 
given class are remote. However, complications arise 
when some objects of a class may be local, others 
remote, and some may be in the kernel. Topically, 
RPCs employ only one mechanism to marshall a pro- 
cedure call parameters into a message. Using the same 
RPC mechanism to marshal] a method call to each 
of these different locations imposes unnecessary over- 
head. In contrast, . our proposed implementation is 
both transparent and recontlgurable on the fly. 

Proxies are the ley to our API provisions. 
Shapiro[lO] proposed proxies, which act as the local 
representatives for groups of distributed servers. We 
interpret a proxy in more general terms as the local 
representee of any object which exists in a different 
address space. Enhancements may optimise access to 
remote objects, provide support for changing server 
interfaces, and allow instrumentation of server perfor- 



mance. 

The object model in distributed Choices is intro- 
duced in Section 2. We then describe the existing 
Choices API in Section 3. Next, in Section 4 we list 
properties of proxies which are useful in API exten- 
sions. The table driven RPC scheme based on proxies 
is then described in Section 5. Related systems are 
surveyed in section 6. Section 7 highlights our experi- 
ence and discusses future work. 



2 The object model 

In Choices all entities including files, disk par- 
titions, address spaces (domains), processes, CPUs, 
messages, servers, classes, semaphores and timers are 
objects. Objects vary in granularity from fine grain 
(like messages) to coarse grain (domains). Objects 
may be located anywhere in the system and may move. 
Objects communicate with each other by method in- 
vocation. The basic aim of the object model is to pro- 
vide a uniform, object-oriented wa.y of structuring the 
system as well as applications and allow transparent 
access to all objects. 

AH objects are instances of a class belonging to a 
class hierarchy. Objects can be private or proxiable. 
Private objects are accessible only within a domain. 
Proxiable objects may be accessed from other do- 
mains, subject to protection policies and constraints. 
An object is designated as private or proxiable de- 
pending its class. Such a class definition is annotated 
to indicate the methods that may be invoked globally. 
A further annotation permits processes in other do- 
mains to create new instances of the class. References 
to proxiable objects are made available to other do- 
mains by registering the objects with an appropriate 
Name Server . 

Method invocation on objects in Choices is uni- 
form, irrespective of object location. The distribution 
of objects is transparent. A reference to a proxiable 
object is obtained from a NameServer, This reference 
is in the form of a proxy object that represents the 
remote object. A method call on the proxy object is 
transformed into a method call on the proxiable object 
by the proxy object. Figure 1 shows an application 
program accessing the NameServer to find an object 
using the method lookup on the NameServer. The 
NameServer locates and authenticates access to the 
object and then allocates a proxy object to provide a 
local representative of the proxiable object. 

The new operator may be used to create a new ob- 
ject of a proxiable class. The ObjectServer is respon- 
sible for object creation, placement and registration of 
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the object with the NameServer. A simplified view of 
the creation process is shown in Figure 2. 

NameServers and ObjectServtrs are proxiable ob- 
jects. Standard NameServers are accessed using de- 
fault proxy objects that are created when a domain is 
created. Thus, applications can access the Standard 
NameServers without requiring a lookup. 

3 Proxy objects and the kernel 

Proxy objects were introduced into Choices to pro- 
vide applications with a protected, object-oriented in- 
terface to system objects[10]. The ObjectProzy class 
defines proxy objects. Before an application can in- 
voke a method on a system object, it must first obtain 
a proxy object for that object from the name server. A 
method call to an ObjectProzy results in a method call 
to the corresponding system object. The ObjectProzy 
implements an indirection that is used to provide con- 
trolled access to system objects from un trusted appli- 
cations. Besides providing this form of access to exist- 
ing kernel objects,, new kernel objects may be created 
using the C++ new operator. Dykstra shows how the 
proxy interface can be used to extend a Choices class 
hierarchy by subclassing kv:i;el classes in user space[2). 

3.1 Proxy objects 

Applications invoke a method on a kernel object 
by invoking the corresponding method on its Object- 
Prozy^}. Messages sent to an ObjeetProzy are for- 
warded to the corresponding kernel object. Object* 
Proxy s are dynamically created by the kernel dur- 
ing name server lookup of the kernel object and are 
protected by allocating them in application read-only 
memory. An ObjectProzy shares a virtual function ta- 
ble with other similar proxy objects. Each method of 
the proxy object represents a method of the kerne] ob-. 
ject for which it is a proxy. The virtual function table 
entries for an ObjectProzy are identical and reference 
code that forwards the method call to the kernel ob- 
ject within the kernel. 

When invoked, an ObjectProzy method traps into 
the kernel. The trap handler has access to special 
tables which contain information about each kernel 
method and its parameters. The trap handler checks 
the parameters, makes copies of the parameters as nec- 
essary, and calls the actual kernel object methods. If 
proxy objects axe passed as parameters into the ker- 
nel, the parameter mechanism checks whether they 
are valid parameters and replaces the proxy object 
reference with a reference to the actual kernel object. 



When the method call returns, any results are copied 
back into the address space of the application. Any 
pointers to kernel objects that are returned are con- 
verted into pointers to proxy objects. 

3-2 Specifying the application interface 
for kernel objects 

The tool Proxify4-+[2] generates the proxy tables 
that are used by ObjectProzies. It also generates tlTe 
kernel object header files that are used by applications 
to call kernel objects. The source code declarations of 
kernel object methods that are to be exported to ap- 
plications through the API are marked with the key- 
word proxiable (which is an extension to the C++ 
language.) Proxify-f-f is a preprocessor that parses 
the source C++ header files of the kernel' objects and 
compiles proxiable methods into: 

• ProzyTables that are used in the implementation 
of proxy calls and 

• application header files that are linked with appli- 
cations that want access to the proxied methods, 
constructors, and classes of kernel objects. 

The application interface is generated automati- 
cally by running all Choices header files through 
Proxify+4- 1 . 



4 Proxies and distributed systems 

Shapiro introduced proxy objects as local represen- 
tatives of remote services in a distributed system [10]. 
Proxies in Choices are local representatives of objects 
located outside the. address, space _of an application. 
They are allocated ~dyhVm?ci^y"wh~en an application 
"looks up" the address of a proxiable object that 
has been registered with or bound to the nameserver. 
In this scheme, the internal state of a proxy object 
records whether a remote or kernel access mechanism 
is to be used to access a particular object. The param- 
eters of this mechanism can he changed dynamically to 
allow objects to move while they are used. According 
to the proxy principie[10], the proxy is the only visible 
interface to a service. To the application, interacting 
with the proxy is identical to interacting with the ac- 
tual object as if it were a local object. Thus proxies 

'The preprocessor is based on a MPL compiler built at the 
University of Virginia by Ed Loyot and Andrew Grizxuhaw, 
which Has been extended to recognize the new keyword prozi- 
cblt. 
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Figure 1: Object lookup 



allow transparent access to objects located anywhere 
in a distributed system. 

Our design for distributed Choices makes extensive 
use of proxies. In the following discussion, the term 
client refers to an application that invokes a method. 
The term server refers to an object upon which the 
method is being invoked. We extend the basic ser- 
vices proposed for proxies in [10] to include dynamic 
subtyping, multiple versions of server interfaces, in- 
strumentation of client invocations of server methods, 
and local client caching of remote server information. 
We now give a more detailed description of the func- 
tions provided by a proxy. 

Proxies provide three different categories of func- 
tions: essentia], server specific optimizations and ad- 
ditional. 



4.1 Essential functions 



These functions are provided by every proxy. Al- 
though the essential functions are identical for all 
proxies, the implementation of the functions depends 
on server type and location. For example, different pa- 
rameter marshalling schemes are used for local kernel 
services and for remote services. 



Communication with server : This function is 
responsible for transferring data between the proxy 
and the server. Depending on the location of the 
server, this could involve copying data, mapping ad- 
dress spaces to allow data sharing, or passing messages 
over the network. Since the location of the server may 
change after the proxy is allocated in a system imple- 
menting object migration, the ability to reconfigure 
this service dynamically is essential. 

Forwarding and marshalling of requests : A re- 
quest is an invocation of a server method by the client. 
A proxy, as the server's representative, validates the 
request by checking parameters. It also marshalls the 
parameters into a request packet, which is forwarded 
to the server. Machine dependent translations like 
byte ordering are also handled here. 

Protection : Proxies provide protection to both the 
client and the server. The client must be assured that 
the proxy really represents the service it asked for. 
The assurance is given by having a trusted process 
allocate the proxy in a separate unmodifiable address 
space. One way to do this is to use a kernel process and 
to allocate proxies in read-only memory. The access 
rights of clients to invoke methods on the object are 
validated at "look up" time by the nameserver and 
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Figure 2: Object creation 



again at request time. Restricting the client's access to 
a subset of the server interface could be implemented, 
by run time checks during each access. 

Failures : Failure of a client or server is just as dif- 
ficult (or easy) to recover from as client or server fail- 
ures in RPC schemes. Depending on whether just the 
client or the client machine has failed, the failure con 
be detected by the proxy or the server. In either case, 
the server knows of the failure and can handle it ap- 
propriately. Server failure is harder to handle because 
transparency requirements are violated. Aborting the 
client with a special error message is one option. In 
any case, the proxy object provides an agent for at- 
tempting to implement failure detection and/or recov- 
ery as transparently as possible. 



4.2 Server specific optimizations 

Optimizations can improve the performance of a 
method call on a proxy object but may make relia- 
bility more expensive. In general, a proxy object can 
record the parameters of a method call and its re- 
sults and reuse the data to improve performance of 
similar future method calls. More specific optimisa- 
tions include caching file system access requests using 
block or whole tile caching and building a hash ta- 
ble of names requested from name servers. Because 
the application interface is a proxy object, it is easy 
to provide the object with internal state that records 
cached data. The proxy object may also be used to 
support cache coherency and data consistency proto- 
cols. Name server proxies may have buffers to store 
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hints. File server proxies may include cache coherence 
protocols for shared access to files. 

Providing local service : Some requests to remote 
objects may be more easily provided locally by the 
proxy itself. For example, to find oat more informa- 
tion about the class of a proxied service, the class Of 
method, which returns the name of the class of an ob- 
ject, could be implemented by the local proxy object. 

4.3- Additional services 

These services are required only occasionally, for 
example, when measurements on server ntilixation by 
a particular client are being made or when server in- 
terfaces have changed. 

Performance measurements : Measurements of 
server response time and its dependence on client lo- 
cation with respect to the server can be made by in- 
serting instrumentation code in the proxy. Adding 
a counter instrument and a timer instrument to the 
proxy allows, for example, the measurement of the 
number of requests to the server from an application 
and the average response time. 

Server versions : Server interfaces may change, ei- 
ther by the addition of more public methods or by al- 
tering the type or number of parameters to a method. 
In the latter case, clients thai use the old interface 
must be recompiled to use the new interface. An al- 
ternative approach is to build proxy objects for both 
interfaces, the proxy object for the old interface con- 
verts method calls to the new interface. Default values 
of missing parameters and type conversions between 
old and new types can be specified. Although there 
are limitations to the translations which are feasible, 
the proxy object removes the need to recompile appli- 
cations because of minor server changes. 



5 Table-driven RPC 

In this section we describe the construction of a 
proxy object mechanism that uses table-driven RPCs 
to provide applications with transparent system-wide 
access to objects. We briefly overview different RPC 
implementations and the limitations of those imple- 
mentations to support a transparent dynamically- 
coniigurable API. Then, we outline our approach. 



5.1 RPC implementations 

Conventional RPC Implementations use stubs to 
implement non-local calls. A stub generator parses 
interface specifications for servers and generates stubs 
which are linked with the applications. A stub im- 
plements a remote procedure call by performing pa- 
rameter format conversions, marshalling the call and 
parameters into a message, and sending the message 
to the appropriate server. The functions performed 
by the stub are determined at compile time and, 
once linked to an application, the application becomes 
bound tp these- functions. The stub approach has the 
following limitations: 

1. Clients can only invoke RPCs on new servers that 
are subclasses of the class for which the stub was 
generated. 

2. The marshalling, format conversion, and any pa- 
rameter checking performed by the stub cannot 
be changed without relinking the client. In par- 
ticular, this becomes a problem for objects with * 
long lifetimes. 

3. Should the client or server move during execution, 
perhaps as a result of dynamic load balancing or 
node failure, the stub cannot be altered dynami- 
cally to improve performance or recover from the 
failure. 

4. For one client, different stubs must be used to 
implement RPCs to different objects of the same 
class if the nature of that access if different. For 
example, the stub may perform caching optimiza- 
tions, provide access to kernel objects, local ob- 
jects in a different virtual memory space or re- 
mote objects. 

5. Should instances of & new version of the class of 
a server replace old instances, new stubs must be 
generated and the clients must.be relinked. 

5.2 Choices proxy RPC 

We make use of. the existing Choices implemen- 
tation of controlled access to kernel objects and ex* 
tend the proxy mechanism to provide transparent ac- 
cess to remote objects. Instead of including all mar- 
shalling and processing code in RPC stubs, we use ta- 
bles to store information about the appropriate mar- 
shalling routines for each parameter in each remote 
method. These dynamic tables define the interface to 
each server. A table entry is associated with each ob- 
ject that is accessible to a client application and the 
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index of the entry is stored in the corresponding proxy 
object. The proxy object uses its index and these ta- 
bles to identify the appropriate parameter maishalling 
functions dynamically. The table-driven approach has 
the following advantages: 

• The tables become a standard interface foi the 
RPC mechanism. They are independent of the 
specification language (for example, they do not 
depend on C++ or Proxify4+.) Different RPC 
parameter specification languages can be used for 
different languages without having to change the 
table format. 

• The tables can be updated dynamically. This 
makes it possible to add new class entries as 
needed. 

• There is no need to recompile the client kernel 
or server kernel if a class definition used as an 
interface for a remote RPC method call changes. 
It is often sufficient to update the tables with the 
new class definition. 

5.3 Implementation details 

The RPC system is built on top of the Choice* mes- 
sage passing system [7]. The proxy object mechanism 
uses information generated by Prcorify++ in the form 
of proxy object tables to implement method calls to 
the objects corresponding to the proxy. A proxy ob- 
ject table includes information describing the methods 
of an object, and is independent of whether the object 
is in the kernel or remote. A few extensions to the 
syntax of headeT files allows the specification of read 
only and write only parameters, array size, levels of 
pointer indirection, and null-terminated strings as pa- 
rameters. This information is needed for marshalling 
arguments to remote methods. The ObjectProzy class 
is subclassed to RemoteProzy for remote access to ob- 
jects and KernelProzy for local access to the kernel. 
All the methods of the parent class are redefined to 
use the appropriate kernel or remote method call in- 
terpretation of the proxy tables. The way in which 
a kernel object is accessed using a KernelProzy has 
already been discussed in Section 3. 

Invoking methods on remote objects : A Re- 
moteProzy object representing a remote object is al- 
located during NameServer lookup on that object. 
All method invocations on the remote object are han- 
dled by the RemoteProzy. When an application in- 
vokes the remote object's method, the correspond- 
ing RemoteProzy object method invokes a generic ta- 



ble interpreter function and passes a reference to the 
proxy table entry for the class and method. Each 
table entry stores information about the appropriate 
marshalling routines for each parameter in the re- 
mote method. The marshalling functions which are 
invoked for each parameter by the table interpreter • 
are machine dependent. For example on slack based 
machines in a network, marshalling involves reading 
each parameter from the stack and packing it into 
the appropriate transmission format. Parameters of 
the same type share the same marshalling function. 
These functions also handle other machinene depen- 
dent translations like byte ordering. The RemoteProzy 
also implements parameter validation and communi- 
cation with the server. The RemoteProzy object has a 
handle to the Message Container of its, server. The 
RemoteProzy object also keeps specific information 
about the remote object and it satisfies some requests 
to the remote object locally. The arguments returned 
as a result from a method call aie stored back using a * 
similar approach. 

Creating new remote objects : The new opera- 
tor is used by applications to create new instances of 
any class. The result of a new operation is the creation 
of a new object in the local address space, in the kernel 
or in a remote address space. Local classes are han- 
dled by the language implementation and the handling 
of kernel classes is described by Dykstra[2]. Creating 
objects of remote classes is handled by the Object- 
Server object. A new on a remote class is handled 
by invoking the construct method of the appropri- 
ate ObjeciServer. The correct ObjectServer object is 
located by NameServer lookup. The construct rou- 
tine is then invoked just like any other remote method 
invocation. This routine creates a new object in the 
correct domain, binds a server process to it if needed, 
registers the object with the NameServer, creates a 
RemoteProzy object for the remote object and returns 
it to the application. All remotely accessible objects 
have to be created using the ObjectServer, though once 
created, no further interaction with this object is re- 
quired. 

Creating servers : Services are exported by anno- 
tating class declarations with the proxiable keyword. 
A service class declares all methods that it wants to 
be globally accessible as proxiable. Instances of such 
a class can be created by applications if the class con- 
structor is proxiable. If this is not the case, objects 
of this class can be created by the owner only. Any 
object of a service class is a server. All objects of 
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a service class are advertised in the NamcServcr. A 
seiver object is bound to a process which services all 
requests for method invocations on the object. A pro- 
cess may serve more than one object. Registration 
with the NameServer and binding a process is han- 
dled at object creation time by the ObjectServer. 



6 Related work 



Proxies were defined by Shapiro[10] as server repre- 
sentatives. He specified several desirable functions in- 
cluding access protocols, buffering, access control and 
communication functions. We use proxies as an inter- 
face to all objects not in the local address space. The 
use of proxies is transparent to the programmer. We 
also define some additional properties which might be 
useful, including failure handling, performance mea- 
surements and server versions. 

Different mechanisms including ports, stubs, ca- 
pabilities and virtual memory have been proposed 
as abstractions to simplify distributed programming 
and provide protected and transparent access to re- 
mote services(8, 3, 5]. Proxies subsume the func- 
tionality provided by ports, stubs and capabilities [10]. 
Comandos[5, 6] uses references to identify objects and 
uses a run time support library which is linked to user 
programs to perform object invocations. Specifically 
the invoke library function, which takes the method 
name as a parameter is used to implement a method 
call. Proxies are used in the special case of communi- 
cating with generic servers, while service requests to 
dedicated servers are handled by the system taking ad- 
vantage of virtual memory. Our approach uses proxies 
for all non-local method invocations. Optimization of 
access depending on the location of the object, is han- 
dled by customixing the proxy. This provides a more 
uniform interface. 

The remote procedure call (RPC) is a common ab- 
straction used to provide easy and transparent access 
to services in a distributed system[4]. A common way 
of implementing RPC is by providing stubs for each of 
the remote functions in a library which is then linked 
with applications[ll]. We use a table-driven approach 
to implement our method call facility which provides 
greater flexibility in customizing access and making 
changes to the server interface without sacrificing ef- 
ficiency. 



7 Conclusions 

We have described the design of a table-driven 
proxy object RPC mechanism for Choices which pro- 
vides controlled access to kernel and remote objects. 
The facility uses proxy objects to provide a trans- 
parent and reconfignrable application interface for a 
distributed system containing kernel and server ser- 
vice objects. The applications interface allows mobile 
services. It encourages microkernel experimentation 
since services can be relocatable transparently from 
the kernel to remote hosts without requiring applica- 
tions to be recompiled or relinked. The scheme also 
permits proxy objects to perform local optimisations 
by, for example, caching and to support versioning 
of classes defining server objects by converting old 
method protocols to the new version of the protocol 
We are examining the possibility of automating some 
optimisations and version support and adding some 
form of grouping or clustering to the Choice* object 
model. 
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